Repository navigation
Merge upstream DeepGEMM through PR #432 - #15
Merged
Merged
Conversation
* Public release 26/09 * Update News
Integrate upstream 66081d4 while preserving the fork training, GeGLU, strided PSUM, and SM-headroom features. Port custom launchers to DeepJIT and adapt backward storage and scheduling to the upstream interfaces. Validation: extension build, six CPU API regressions, and 26 offline SM100 CUDA variants. GPU numerical and distributed execution remain unverified on this host.
murphymatt
marked this pull request as ready for review
September 10, 2026 09:04
There was a problem hiding this comment.
Risk: high. Left a non-blocking comment and did not approve: this upstream DeepJIT/runtime merge changes public GEMM, attention, and MoE APIs with a large blast radius, so it is above the low-risk auto-approval threshold. Human review is needed; no reviewers were assigned.
Sent by Cursor Approval Agent: Pull Request Approver
YLGH
approved these changes
Sep 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Merge upstream DeepGEMM through
66081d4c9c7d7c44f13fea402e5b622aa0f409c2(PR #432), including the July release and tensormap race fix. The merge retains all six fork commits and ports the fork's training paths onto upstream's DeepJIT runtime, buffer layout, and scheduler.test_mega_moe.pyand move the fork's training suite totest_mega_moe_training.py. Add CPU API regressions and a reproducible offline CUDA compilation suite.Validation passed with GCC 13 and the CUDA/PyTorch development container:
Draft because this host has no working NVIDIA driver: GPU numerical correctness, distributed synchronization, and performance tests have not run. Shared experts combined with training saves or nondefault route/combine modes are explicitly rejected. Existing raw buffers must be reallocated for the new layout and 1920-token alignment; explicit ring capacities must satisfy the new scheduler's estimate. See
docs/fork-upstream-sync.mdfor compatibility details and validation commands.Please use a merge commit when landing this sync to retain upstream ancestry.
Note
High Risk
Large merge replacing the JIT/runtime layer and extending MoE, attention, and GEMM APIs on security- and performance-critical GPU paths; callers must re-init JIT, refresh symmetric buffers, and validate env/cache behavior.
Overview
This sync lands upstream DeepGEMM through PR deepseek-ai#432: DeepJIT replaces the in-tree JIT stack and
{fmt}(newthird-party/deep_jitsubmodule, localcompiler.hpp/cache.hppremoved).initnow only takes the library root;DG_JIT_*env vars delegate toDJ_JIT_*, and device/heuristics go throughjit/runtimeinstead ofdevice_runtime.Attention / indexer: MQA paths are refactored (dtype-driven FP8/FP4, optional
schedule_meta, logits cleaning fused in-kernel). New sparse and paged sparse MQA APIs plus metadata helpers and CUDA-graph-friendly workspaces.GEMM / einsum: Optional
alphaon FP8/FP4 and BF16 GEMMs; K-grouped FP4 contiguous GEMM; cuBLASLt NVFP4 GEMM and batchedsyrk/symm; FP8 BMM/einsum can emit FP8 + UE8M0 SFs. Deterministic mode toggles cuBLASLt vs custom kernels;use_deterministic_algorithmsis exposed.Mega stack: Mega Gate (fused BF16 gate + top-k), Mega mHC (shifted/normal mHC + RMSNorm + optional FP8 outputs), and Mega MoE v3 buffers with shared experts, FP8×FP8 routed weights (
mma_type), scheduler-based ring sizing, andget_block_m_for_mega_moe. Paged MQA allowsblock_kv32/64/128.Docs/requirements tighten to CUDA 12.9+, PyTorch 2.3+, C++20
<format>, andlibdw-dev.Reviewed by Cursor Bugbot for commit 38eb89f. Bugbot is set up for automated code reviews on this repo. Configure here.